Extending Local Learners with Error-correcting Output Codes Extending Local Learners with Error-correcting Output Codes

Error-correcting output codes (ECOCs) represent classes with a set of output bits, where each bit encodes a binary classiication task corresponding to a unique partition of the classes. Algorithms that use ECOCs learn the function corresponding to each bit, and combine them to generate class predictions. ECOCs can reduce both variance and bias errors for multiclass classiication tasks when the errors made at the output bits are not correlated. They work well with algorithms that eagerly induce global classiiers (e.g., C4.5) but do not assist simple local classiiers (e.g., nearest neighbor), which yield correlated predictions across the output bits. This is distressing because local learning algorithms for classiication are preferable to global classiiers for some types of applications. We show that the output bit predictions of local learners can be decorrelated by selecting diierent features for each bit. We present promising empirical results for this combination of ECOCs, nearest neighbor , and feature selection. We also describe modiications to the schemata racing algorithm for feature selection that improve its ability to retrieve good feature subsets in this context. Abstract. Error-correcting output codes (ECOCs) represent classes with a set of output bits, where each bit encodes a binary classiication task corresponding to a unique partition of the classes. Algorithms that use ECOCs learn the function corresponding to each bit, and combine them to generate class predictions. ECOCs can reduce both variance and bias errors for multiclass classiication tasks when the errors made at the output bits are not correlated. They work well with algorithms that eagerly induce global classiiers (e.g., C4.5) but do not assist simple local classiiers (e.g., nearest neighbor), which yield correlated predictions across the output bits. This is distressing because local learning algorithms for classiication are preferable to global classiiers for some types of applications. We show that the output bit predictions of local learners can be decorrelated by selecting diierent features for each bit. We present promising empirical results for this combination of ECOCs, nearest neighbor, and feature selection. We also describe modiications to the schemata racing algorithm for feature selection that improve its ability to retrieve good feature subsets in this context.